Distributed sensing and video capture system and apparatus

ABSTRACT

Systems and apparatus for sensing and video capture include at least one camera with an optical sensor that captures video image data of a first sampling rate. An auxiliary sensor captures auxiliary data at a second sample rate. A processor is communicatively connected to the optical sensor and auxiliary sensor. The processor transmits video image data captured at the first sample rate auxiliary sensor data captured at the second sampling rate across a data connection to a centralized computer that receives the video image data and the auxiliary sensor data and operate to present the video image data and the auxiliary sensor data on a graphical display.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation of U.S. application Ser. No.15/701,024, flied Sep. 11, 2017, which application was published on Jan.11, 2018, as US2018/0013901, which application further is a continuationof U.S. patent application Ser. No. 15/473,244, filed Mar. 29, 2017,which application was published on Jul. 20, 2017, as US2017/0208189,which application further is a continuation of U.S. patent applicationSer. No. 14/280,005, filed May 16, 2014, which application was publishedon Nov. 20, 2014, as US2014/0340533, and further claims priority of U.S.Provisional Patent Application No. 61/823,948, filed on May 16, 2013,the contents of which are hereby incorporated herein by reference intheir entireties.

BACKGROUND

Currently closed circuit television (CCTV) systems include a pluralityof video cameras connected by cabling to a central point at a facilityor to a remote location where the video is recorded and/or displayed.CCTV systems are used for surveillance, security, and the collection ofenterprise data. The installation costs for the data and power cablingfor each of the video cameras in the system can approach or exceed thecost of the cameras themselves. Therefore, CCTV systems represent asignificant investment for a facility.

BRIEF DISCLOSURE

An exemplary embodiment of a system for sensing and video captureincludes at least one camera. The at least one camera includes anoptical sensor, at least on auxiliary sensor, and a processor. A dataconnection is connected to the processor for transmission of data fromthe processor. A centralized computer receives the data from theprocessor across the data connection. The processor acquires andtransmits video image data from the optical sensor at a first samplerate. The processor periodically acquires and transmits auxiliary sensordata from the auxiliary sensor. The centralized computer receives thevideo image data and the auxiliary sensor data across the dataconnection.

An exemplary embodiment of a digital video camera includes an opticalsensor that captures video image data at a first sampling rate. Thedigital video camera includes an auxiliary sensor that capturesauxiliary sensor data at a second sampling rate. A processor iscommunicatively connected to the optical sensor and to the auxiliarysensor. The processor transmits the video image data captured at thefirst sampling rate and the auxiliary sensor data captured at the secondsampling rate.

An additionally exemplary embodiment of a system for sensing and videocapture includes a plurality of cameras. Each camera includes an opticalsensor, at least one auxiliary sensor, a processor, and a power supply.The optical sensor acquires video image data. The at least one auxiliarysensor acquires auxiliary data. The processor receives and processes thevideo image data from the optical sensor and the acquired auxiliary datafrom the at least one auxiliary sensor. The power supply provides powerto the optical sensor, at least one auxiliary sensor, and a processor.At least one data connection is connected to each of the plurality ofcameras. In each of the plurality, of cameras, the data connection isconnected to the processor and to the power supply. A centralizedcomputer receives the digitized data from the processor across the dataconnection. The power supply in each camera of the plurality of camerasconverts power from the data connection to provide power to theprocessor, optical sensor and at least one auxiliary sensor. Theprocessor transmits the video data at a first sample rate andperiodically acquires and transmits the auxiliary data at a secondsampling rate across the data connection to the centralized computer.The first sampling rate is greater than the second sampling rate.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic representation of an exemplary embodiment of acamera as disclosed herein.

FIG. 2 is an exemplary embodiment of a system for sensing and videocapture.

FIG. 3 is a system diagram of an exemplary embodiment of a computingsystem.

DETAILED DISCLOSURE

The introduction of internet protocol (IP) cameras has somewhatsimplified the installation of CCTV systems as the output of the IPcamera is digital and can travel over the same cabling used to power thecamera using a technique called Power over Ethernet (POE). The IP cameraincludes a processor that is programmed to capture and perform basicvideo processing before the image data is packaged and sent to acentralized computer. The centralized computer may be on-site orremotely located.

In embodiments as disclosed in further detail herein, the disclosed IPcamera includes one or more auxiliary sensors, exemplarily for sensingtemperature, humidity, and/or acceleration. The processor processes theinformation obtained from these sensors at a lower sampling rate thanthe video signals and periodically multiplexes packages with the datafrom these auxiliary sensors to the centralized computer where theresulting data can be presented, processed, and/or stored.

FIG. 1 depicts an exemplary embodiment of an IP camera 10 as may be usedin the embodiment of the system as disclosed herein. The camera 10includes a housing 12 that contains and protects the electroniccomponents found therein. Video capture functions of the camera 10 arefulfilled by a lens 14 and an optical sensor 16. The optical sensor 16provides the captured image data to a processor 18 which either includesor is connected to a computer readable medium (not depicted) for storingcomputer readable code that upon execution by the processor causes theprocessor to execute the functions and operations as disclosed infurther detail herein, including the operation of the optical sensor 16to acquire images at a specified sampling rate.

The camera 10 further includes a power supply 20 which as previouslydisclosed may operate to provide power to the camera 10 through POE. Itwill be recognized that in alternative embodiments, the power supply maybe other forms of power supplies including, but not limited to utilityelectrical power, battery power, solar power cells, or a converter ofwirelessly transmitted power. However, it is recognized that in somesecurity settings, CCTV systems need be operated with a dedicatedhardwired communication and/or power. Therefore, in such embodiments,POE offers a solution to provide communication and power to the cameras10 of a CCTV system. In an embodiment, the processor 18 and power supply20 are connected to an Ethernet cable 22 which is connected to thecentralized computer. In still further embodiments, particularly thosethat do not operate on POE, the processor 18 may be communicativelyconnected to the centralized computer in other manners, includingwireless networks.

The IP camera 10 further includes one or more auxiliary sensors 24,which in a exemplary embodiment are microelectromechanical sensors(MEMS) 24 which will be described in further detail herein. Non-limitingexamples of such MEMS that may be used in embodiments disclosed hereininclude light sensors, magnetic field sensors, pressure sensors,chemical sensors, accelerometers, temperature sensors, and humiditysensors; however, this is not intended to be limiting on the types ofMEMS that may be incorporated into the camera 10 in accordance with thepresent disclosure. However, it will be recognized that other types ofsensors besides MEMS may be used, including sensors for light, magneticfield, pressure, chemical, acceleration, temperature and humiditysensing.

Auxiliary sensors 24 are connected to the processor 18 and provideadditional sensing and capability to the camera 10. In embodiments asdisclosed herein, the auxiliary sensors 24 operate at a lower samplerate than the optical sensor 16. Although in alternative embodiments,the auxiliary sensors 24 may sense the physical characteristic at ahigher sample rate although the processor 18 only samples or receivessuch samples from the auxiliary sensors 24 at a lower sample rate. In anon-limiting example, the optical sensor 16 may be multiplexed with theone or more auxiliary sensors 24 such that the processor 18 obtainssamples far more frequently from the optical sensors 16, than any of theauxiliary sensors 24. In an embodiment, the purpose of this is tomaintain the quality of the video capture and desired refresh rate ofthe video capture system, while providing additional sensingcapabilities in the background of the video capture process.

The processor 18 operates to process the signals obtained from theauxiliary sensor 24 to create auxiliary data. The processor 18 thenpackages the required auxiliary data in a suitable format fortransmission on the Ethernet cable 22 in between packages of video data.

Further features of the disclosed system will be highlighted below withrespect to the incorporation and use of various auxiliary sensors,including MEMS, in exemplary embodiments.

FIG. 2 depicts an exemplary floor plan layout of a CCTV system 30. TheCCTV system 30 includes a plurality of cameras 10, which are exemplarilyIP cameras 10 as described above. The plurality of cameras 10 areconnected to a centralized computer 32 by one or more Ethernet cables22.

FIG. 2 exemplarily depicts a basic layout of a bank, although suchlayout is merely for contextual purposes and is not intended to belimiting in any way on the function, layout, or facilities within whichsuch a CCTV system may be established. In the layout depicted in FIG. 2,features include a plurality of offices 34, exterior doors 36, aplurality of teller stations 38, a vent 40 of an environmental controlsystem, exemplarily an HVAC system, and an automated teller machine(ATM) 42. As can be seen in FIG. 2, the cameras 10 are distributedthroughout the CCTV system such as to achieve a desired video capturecoverage of the area. This is achieved by a wide distribution of cameras10 throughout the facility, but also the IP cameras are positioned atspecific sensitive areas such as the teller stations 38 or the ATM 42.

In a non-limiting embodiment wherein the auxiliary sensor is atemperature sensor, this distributed network of temperature sensors canfurther provide temperature information across the facility. This canconstructively be used as a more distributed thermostat which, ifintegrated with environmental controls at the centralized computer 32,can enable more efficient operation of the environmental control system.In embodiments, the centralized computer 32 can automatedly send controlsignals to environmental control apparatus 22 (e.g. heater, airconditioner, humidifier) in response to the received auxiliary data(e.g. temperature). For example, customer or employee complaints oftemperatures either being too hot or too cold can be empiricallyverified and operation of the environmental control system adjusted toaddress the problem. An example of an camera 10 with temperature sensorat or near an HVAC vent 40, can confirm HVAC operation to determine thetemperature of the air blown into the facility is of a desiredtemperature for heating or cooling purposes.

The temperature at various locations in the facility does not changevery rapidly, and therefore is not required to be sampled exemplarilymultiple times per second, but rather can be sampled with less frequencyexemplarily every 10 seconds, 30 seconds, once a minute, or even lessfrequently, although these sampling intervals are merely exemplary andare not intended to be limiting on the scope of the disclosure. Thisinformation on the temperature gradient within the facility can be usednot only for specific environmental controls, but also can provideenvironmental information to facilitate improvements or remodeling toimprove energy efficiency. For example, it may be identified thatlocations near the doors 36, or along a particular wall are colder orhotter than the rest of the facility and may be so at specific times ofday. Therefore these areas may be identified for structural or energyefficiency improvement.

In still further embodiments, the incorporation of auxiliary sensor 24that include a magnetic field sensor or a chemical sensor can provideadditional safety and/or security capability to monitoring functionsperformed by the centralized computer 32. In an embodiment wherein theauxiliary sensor is a magnetic field sensor, auxiliary data indicativeof sensed magnetic fields can be used by the centralized computer toidentify intention or incidental sources of magnetic interference. Thisidentification can improve security measures by identifying potentialthreats. Still further embodiments facilitate remodeling or layout ofequipment or systems such as computers, televisions, power sources, orcommunication infra-structure so as to limit or minimize magneticinterference. Embodiments that incorporate chemical sensors can acquireand provide auxiliary data to the centralized computer 32 indicative ofcarbon monoxide presence or concentration. This can add safety featuressuch as carbon monoxide detection into the existing system. In a furtherembodiment, detecting of chemical concentrations, exemplarily near anHVAC vent may provide an additional indication of the operation ormaintenance needs of the HVAC system, for example if exhaust,particulates, or pollution is detected in the air entering the room.

In still further embodiments, the auxiliary sensors may be a humiditysensor and similar to the operation and use of embodiments withtemperature sensors described above, the relative humidity within afacility may be measured or monitored and appropriate adjustments andcontrols provided to the environmental control system 44 may be made toreach a desired humidity level within a facility. Alternatively, such amapping of the humidity within a facility may further direct maintenanceor remodeling improvements that could be made to better control orachieve a desired humidity.

In an alternative embodiment, the auxiliary sensor may be anaccelerometer that provides exemplarily three axis information on theangular attitude of the IP camera 10. Similar to that described abovewith respect to the temperature sensor, the angular attitude of the IPcamera need not be sampled frequently and therefore can be easilyincorporated into the data stream which is primarily video data fromeach of the IP cameras to the centralized computer 32 along the Ethernetconnection 22. Non-limiting examples of advantages that can be achievedthrough the incorporation of the accelerometer can include assistance ininstallation, whereby once powered on, the IP camera 10 may provideauxiliary data indicative of the angular attitude of the camera suchthat an installer or technician can confirm proper placement of thecameras. Similarly, during operation, relatively infrequent sampling ofthe angular attitude of the cameras can confirm that each of the camerasare providing the desired field of view.

In a more specific embodiment, a camera 10 is either incorporated intoor associated with a device such as an. ATM 42. In order for the camera10 to effectively capture the face of a user of the ATM, the camera 10may require a very specific angle. It would be undesirable for a personof malintent to damage or reposition the camera, such that the camera isnot pointed in the correct monitoring direction and thus does notfulfill its intended purpose. The detection of such a misalignment ofthe angular attitude of the IP camera can then be readily identifiedbased upon the periodic output of the accelerometer and maintenance orsecurity personnel can be dispatched to address the issue and repositionthe camera.

In a still further embodiment, one or more cameras 10 located in aposition at or about a door 36 or other entrance may be used to track orcount the number of people that enter or exit a specific area. Softwarealgorithmic analysis of the images captured by the cameras by thecentralized computer can be used to identify, track, and/or count peopleas they enter or exit a room or area. As a part of these processes, somemay require the angular attitude of the camera in order to properlyidentify people in the captured video image and to determine thedirection of movement of the people in the video images. If a camera'sangular attitude changes for any reason, this information may notaccurately reflect the angle at which the video images are capturedwhich could introduce error into the counting or tracking functions.Therefore, by periodically sampling the actual angular attitude of thecamera as determined by the accelerometer, the actual angle of thecamera can be used for more accurate determinations using such softwareprograms.

FIG. 3 is a system diagram of an exemplary embodiment of a computingsystem 200 as may be used in embodiments to implement the processor 18of the IP camera 10 or the centralized computer 32. The computing system200 is generally a computing system that includes a processing system206, storage system 204, software 202, communication interface 208 and auser interface 210. The processing system 206 loads and executessoftware 202 from the storage system 204, including a software module230. When executed by the computing system 200, software module 230directs the processing system 206 to operate as described in herein. Itwill be recognized that implementations of the computer system 200 foruse as the processor in the IP camera may vary greatly from animplementation used as the centralized computer. Such an implementationas the processor in the IP camera may be a specific use computer orintegrated circuit with all of the components of the computing system200. An implementation of the computing system 200 as the centralizedcomputer 32 may be a general purpose computer that executes a pluralityof software modules, in some embodiments simultaneously, exemplarily tocarry out video analysis, people counting, HVAC or other environmentalcontrol system control, and analysis of other received auxiliary datafrom the IP cameras.

Although the computing system 200 as depicted in FIG. 3 includes onesoftware module in the present example, it should be understood that oneor more modules could provide the same operation. Similarly, whiledescription as provided herein refers to a computing system 200 and aprocessing system 206, it is to be recognized that implementations ofsuch systems can be performed using one or more processors, which may becommunicatively connected, and such implementations are considered to bewithin the scope of the description.

The processing system 206 can include a microprocessor and othercircuitry that retrieves and executes software 202 from storage system204. Processing system 206 can be implemented within a single processingdevice but can also be distributed across multiple processing devices orsub-systems that cooperate in executing program instructions. Examplesof processing system 206 include general purpose central processingunits, application specific processors, and logic devices, as well asany other type of processing devices, combinations of processingdevices, or variations thereof.

The storage system 204 can include any storage media readable byprocessing system 206, and capable of storing software 202. The storagesystem 204 can include volatile and non-volatile, removable andnon-removable media implemented in any method or technology for storageof information, such as computer readable instructions, data structures,program modules, or other data. Storage system 204 can be implemented asa single storage device but may also be implemented across multiplestorage devices or sub-systems. Storage system 204 can further includeadditional elements, such a controller capable of communicating with theprocessing system 206.

Examples of storage media include random access memory, read onlymemory, magnetic discs, optical discs, flash memory, virtual andnon-virtual memory, magnetic sets, magnetic tape, magnetic disc storageor other magnetic storage devices, or any other medium which can be usedto store the desired information and that may be accessed by aninstruction execution system, as well as any combination or variationthereof, or any other type of storage medium. In some implementations,the storage media can be a non-transitory storage media.

User interface 210 can include a mouse, a keyboard, a voice inputdevice, a touch input device for receiving a gesture from a user, amotion input device for detecting non-touch gestures and other motionsby a user, and other comparable input devices and associated processingelements capable of receiving user input from a user. In embodiments,the user interface 210 operates to present and/or to receive informationto/from a user of the computing system. Output devices such as a videodisplay or graphical display can display an interface further associatedwith embodiments of the system and method as disclosed herein. Speakers,printers, haptic devices and other types of output devices may also beincluded in the user interface 210. It is understood that embodiments ofthe computing system implementing portions of the IP camera may notinclude a dedicated user interface and rather user input controls may bereceived through the communication interface 208, and exemplarilyentered by a user at the centralized computer 32 described above.

As described in further detail herein, the computing system 200 receivesand transmits data through the communication interface 208. Inembodiments, the communication interface 208 operates to send and/orreceive data to/from other devices to which the computing system 200 iscommunicatively connected. In an embodiment of the computing system 200implementing the centralized computer, original video data may bereceived from the IP cameras. The original video data is exemplarilystored at a computer readable medium which may be remotely located fromthe computing system. Additionally, the computing system 200 may receivethe auxiliary data from the plurality of IP cameras and the auxiliarydata is also stored at a computer readable medium. The computing system200 executes the at least one application module 230 to carry out thesecurity, safety, environmental monitoring or environmental controlfunctions as disclosed herein. From this processing of the video and/orauxiliary data the computing system 200 can operate an environmentalcontrol or HVAC system or provide notices, alerts, or visual datapresentations on a graphical display of the user interface 210 to reportthe MEMS data or identified abnormalities, unexpected occurrences, oralarm conditions determined by the computing system in the MEMS data.

This written description uses examples to disclose the invention,including the best mode, and also to enable any person skilled in theart to make and use the invention. The patentable scope of the inventionis defined by the claims, and may include other examples that occur tothose skilled in the art. Such other examples are intended to be withinthe scope of the claims if they have structural elements that do notdiffer from the literal language of the claims, or if they includeequivalent structural elements with insubstantial differences from theliteral languages of the claims.

What is claimed is:
 1. A system for sensing and video capture, thesystem comprising: a camera at a facility, the camera including anoptical sensor capable of capturing video image data, an auxiliarysensor capable of capturing non-image data, and a processor thatacquires and transmits the video image data from the optical sensor andacquires and transmits the auxiliary sensor data from the auxiliarysensor; a data connection connected to the processor of the camera forthe transmission of data from the processor; and a centralized computerlocated at a different location from the camera that receives the videoimage data and auxiliary sensor data from the processor of the cameraacross the data connection and processes the video image data andauxiliary sensor data, wherein the video image data is acquired from theoptical sensor and transmitted at a first sampling rate and theauxiliary data is acquired from the auxiliary sensor and transmitted ata second sampling rate wherein the first sampling rate is greater thanthe second sampling rate, such that the processor acquires samples morefrequently from the optical sensor than from the auxiliary sensor. 2.The system of claim 1, wherein the auxiliary sensor comprises at leastone of: a temperature sensor, a magnetic field sensor, a light sensor, achemical sensor, an accelerometer, a humidity sensor, and a pressuresensor.
 3. The system of claim 1, wherein the first sampling rate isspecified by a user.
 4. The system of claim 3, wherein the secondsampling rate is determined by the first sampling rate.
 5. The system ofclaim 1, wherein the second sampling rate is determined based on thetype of auxiliary sensor.
 6. The system of claim 1, wherein theauxiliary sensor is a temperature sensor and the auxiliary data is atemperature measurement.
 7. The system of claim 1, wherein the sensor isa chemical sensor and the auxiliary data further includes an indicationof a chemical concentration, wherein the centralized computer processesthe indication of the chemical concentration to determine theoperational status of the environmental control system.
 8. The system ofclaim 7, wherein the chemical sensor is located near an HVAC vent andthe chemical sensor is configured to determine the chemicalconcentration of air blown into the facility to confirmation operationof an HVAC system.
 9. The system of claim 1, wherein the auxiliarysensor is an accelerometer and the auxiliary data is an orientation ofthe camera.
 10. The system of claim 9, wherein the centralized computerprocesses the received video data from the camera to identify people inthe video data and count the identified people in the video data,wherein the centralized computer uses the received orientation of thecamera to refine the identification of people in the video data.
 11. Thesystem of claim 9, wherein the centralized computer receives theorientation of the camera and compares the received orientation of thecamera to a target orientation and produces an indication if the camerais in an expected position.
 12. An IP camera for a CCTV system,comprising: an optical sensor that captures video image data at a firstsampling rate; an auxiliary sensor that captures auxiliary sensor dataat a second sampling rate; a processor communicatively connected to theoptical sensor and the auxiliary sensor, wherein the processor acquiresand transmits the video image data captured at the first sampling rateand acquires and transmits the auxiliary sensor data captured at thesecond sampling rate such that the processor acquires samples morefrequently from the optical sensor than from the auxiliary sensor,wherein the second sampling rate is determined based on the type ofauxiliary sensor.
 13. The IP camera of claim 12, wherein the firstsampling rate is specified by a user.
 14. The IP camera of claim 13,wherein the second sampling rate is determined by the first samplingrate.
 15. The IP camera of claim 12 such that a greater portion of abandwidth transmitted from the processor is dedicated to thetransmission of the video image data compared to the transmission of theauxiliary sensor data.
 16. The IP camera of claim 15, wherein theprocessor multiplexes the video image data at the first sampling ratewith the auxiliary sensor data at the second sampling rate.
 17. The IPcamera of claim 12, wherein the auxiliary sensor comprises at least oneof: a temperature sensor, a magnetic field sensor, a light sensor, achemical sensor, an accelerometer, a humidity sensor, and a pressuresensor.
 18. The IP camera of claim 13, further comprising: a powersupply configured to connect to a data connection wherein the powersupply uses power over Ethernet (POE) to provide power to the processor,the optical sensor and the at least one auxiliary sensor.
 19. A systemfor sensing and video capture, the system comprising: a camera at afacility, the camera comprising: an optical sensor that acquires videoimage data; an auxiliary sensor acquires auxiliary data that is notimage data, wherein at least one auxiliary sensor is an accelerometerthat acquires auxiliary data describing the camera's alignment; aprocessor that receives and processes the video image data from theoptical sensor and the acquired auxiliary data from the auxiliarysensor; and a power supply that provides power to the optical sensor,the auxiliary sensor, and the processor; a data connection connected tothe camera, wherein the data connection is connected to the processor ofthe camera and to the power supply; and a centralized computer spacedapart from the camera that acquires data from the processor across thedata connection, wherein the video image data is acquired at a firstsampling rate and the auxiliary data is acquired at a second samplingrate, the first sampling rate greater than the second sampling rate,such that the processor acquires samples more frequently from theoptical sensor than from the auxiliary sensor; wherein the power supplyconverts power from the data connection to provide power to theprocessor, the optical sensor, and the accelerometer, wherein thecentralized computer processes the auxiliary data to determine thealignment of the camera and generates an alert when a camera ismisaligned; and wherein the centralized computer processes the auxiliarydata to determine the angular attitude of the camera and provides theangular attitude of the camera to image analysis software.